Optimal sequential probability assignment for individual sequences - Information Theory, IEEE Transactions on
نویسنده
چکیده
The problem of sequential probability assignment for individual sequences is investigated. We compare the probabilities assigned by any sequential scheme to the performance of the best “batch” scheme (model) in some class. For the class of finite-state schemes and other related families, we derive a deterministic performance bound, analogous to the classical (probabilistic) Minimum Description Length (MDL) bound. It holds for “most” sequences, similarly to the probabilistic setting, where the bound holds for “most” sources in a class. It is shown that the bound can be attained both pointwise and sequentially for any model family in the reference class and without any prior knowledge of its order. This is achieved by a universal scheme based on a mixing approach. The bound and its sequential achievability establish a completely deterministic significance to the concept of predictive MDL.
منابع مشابه
Optimal sequential probability assignment for individual sequences - Information Theory, IEEE Transactions on
The problem of sequential probability assignment for individual sequences is investigated. We compare the probabilities assigned by any sequential scheme to the performance of the best “batch” scheme (model) in some class. For the class of finite-state schemes and other related families, we derive a deterministic performance bound, analogous to the classical (probabilistic) Minimum Description ...
متن کاملOn optimal sequential prediction for general processes
This paper considers several aspects of the sequential prediction problem for unbounded, non-stationary processes under p-th power loss lp(u, v) = |u−v|, 1 < p < ∞. In the first part of the paper it is shown that Bayes prediction schemes are Cesaro optimal under general conditions, that Cesaro optimal prediction schemes are unique in a natural sense, and that Cesaro optimality is equivalent to ...
متن کاملAnytime coding on the infinite bandwidth AWGN channel: A sequential semi-orthogonal optimal code
It is well known that orthogonal coding can be used to approach the Shannon capacity of the power-constrained AWGN channel without a bandwidth constraint. This correspondence describes a semi-orthogonal variation of pulse position modulation that is sequential in nature — bits can be “streamed across” without having to buffer up blocks of bits at the transmitter. ML decoding results in an expon...
متن کاملUniversal linear least squares prediction: Upper and lower bounds
We consider the problem of sequential linear prediction of real-valued sequences under the square-error loss function. For this problem, a prediction algorithm has been demonstrated [1]–[3] whose accumulated squared prediction error, for every bounded sequence, is asymptotically as small as the best fixed linear predictor for that sequence, taken from the class of all linear predictors of a giv...
متن کاملDensity estimation from an individual numerical sequence
This paper considers estimation of a univariate density from an individual numerical sequence. It is assumed that (i) the limiting relative frequencies of the numerical sequence are governed by an unknown density, and (ii) there is a known upper bound for the variation of the density on an increasing sequence of intervals. A simple estimation scheme is proposed, and is shown to be L1 consistent...
متن کامل